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ABSTRACT 

Four papers on reading in a second language are 
reviewed and discussed, rhey include a study of assessment and the 
difficulties inherent in the measurement of a construct as complex as 
reading comprehension. The other three deal with variables 
facilitating comprehension in reading. One supports an interactive 
model of reading rather than a skills-based model for promoting 
reading success. Another suggests that texts providing better 
contextual and linguistic clues would stimulate better comprehension 
and recall, and the third proposes training in topical structure 
analysis as a means for improving comprehension. (MSE) 



******************************************************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 
*********************************************************************** 



ERLC 



oo 

Q 

TESOL '8B Colloquiua on Research in Reading in a Second Language: 

Discussion Paper 



■^■•i|i!!!i!i;!!:i!jMVii.!^r: . 




I'i 



l;iiik lM;Jnilifci!'h.lt!ith.. 




AndreM D. Cohtn 
HebreN University of Jerusalem 

Revised Version, April 19BB 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BE^N GRANTED BY 



i^ATERIAL IjUS BE^I 



TO TMt EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) - 



us OCFAMTMCNTOFEOUCATIOM 

Office o( Educational R«March and lmprov«meni 
EDUCATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 

Thia document haa been reproduced aa 
received from the person or offlanizalion 
originating it 
• Minor Changes have been made to improve 
reproduct'on quality 

a Poinis of view of opinions stated m this docu- 
ment <*o not necessarily represent official 

06RI position 0' policy 



00 

to 



ErJc ^ btillCOPY 7AILABLE 



Cohen 1 



Aldirion, J. Charles. Tilting reading coaprehention Bkilli. Lm- 

cafter, Englandt U. of Lancaiter. 
Oevine, Joanne. L2 aodeit of reading and reading strategies. Saratoga 

Springs, NY: Skidaore College. ' 
Barnitz, John 6. I. Speaker, Richard B., Jr. 1988. The roles of context 

and linguistic structure on second language readers' coaprehension 

and retention of poetry. Htm Orleans: Dept. of Curriculue and 

Instruction, U. of New Orleans. 
John Riggles, Jeff Nagoto, Ji. Coady, ( Kouider Hokhtari. Uproving 

reading comprehension through topical structural analysis. Athens. 

OH: Ohio Prograe in Intensive English, Ohio University. 

The first of the four papers I Kill be discussing, Alderson's, 
deals Nith the issue of assessment and the probleas inherent in the 
•easureeent of so coeplex a construct as reading comprehension. The 
other three all deal Nith variables that facilitate coeprehension in 
reading. Bevine claies that those who internalize an interactive 
model of reading are likely to be more successful readers than those 
«.ho have internalized a model that is skills-based. Barnitz and 
Speaker would suggest that texts providing better contextual and lin- 
guistic clues Nill .1,0 stimulate better coeprehension and recall. 
Finally, Rigglis, Hagoto, Coady, and Hokhtari see training in topical 
structure analysis as a means for improving comprehension. 1 «ill 
look at each paper in turn, with an aaphasis on the raising of issues 
intended to promote further discussion. 

ftMffon: Testino Read inn CoaorehenKian 

The first insight that 1 draa from this paper is that there may 
be cognitive hierarchies and implicational scales which are useful for 
the field of reading assessment, but that we have yet io eperational- 
ize these adequately in our language tests. In other aords, it aould 
appear that at the present time, we cannot be sure that any given item 
-ill assess any given skill or level of skill, nor that the answer on 
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ont itt* indicatti to ui hot. anothtr ite. bt .niN.red. H.nc», .y 
conclusion it that tht high/loi. dichoto.y i. net vtry useful. 

Perhaps p.rt of the problea is that soae of the so-called -ena- 
bling skills- are actually .ultif aceted. In other words, the skill 
could take on different characteristics depending on the particular 
reading strategy used to actualize it. Sarig Fol.an (1987), for 
exaaple, present a taiconoay which helps demonstrate that Nbat is 
ostensibly one reading skill, say, 'skiwing a portion of teKt,'« * 
could be realized by .eans of strategies at different taxonoeic 
levels. A given reader could actually be operating on one or all cf 
these Itvels in performing the -.kill.' The levels are technical 
facilitation, clarification and «i.plif Ication, coherence detection, 
and .etacognitive .onitoring. Hence, the psycho.otor act of ski.oing 
text could reflect technical facilitation, in this case, cutting down 
the aeount of detail taken in and the aeount of ti.e expended in read- I 
ing. Such skie.ing through a portion of a text could represent a 
clarification and simplification strategy, as.ii.ing the material were 
deemed redundant or unessential for understanding the essence of the " 
text. The same skimming could reflect coherence detection if the 
focus of the reader .ere, say, on hoN the main idea in each paragraph 
relates to that of the other paragraphs. Finally, the skimming could - 
also be an instance of metacogniti ve monitoring if the reader has 
skimmed, say, after checking for coeprehension and realizing that too 
much time is being spent on this one section of the article with 
dubious results. 



spec 



* Skieming - a) surveying to obtain the gist, b) scanning for 
><ics, is #12 on the Neir 1983 list. ■canning for 
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" " «' "fUr t..... I.. ,„ 

lr»i., t. .b,.i. thi. i.,„..„.„ u th.. .. .... 

thi perciptioni oft 

i> thi tiit conitructor, and 

2) thi tiit takir. 

An- .r. conc.rn.d about the cont.nt validity of our t..t. a. -e 

.hould b.. th.n .ay .i.h to con.id.r th. p.rc.ption. of. 

3) th. out.id. -.Kp.ri.nc.d f.ch.r- judg... a. Ald.r.on and 
others have done in the current study. 

Thi. paper focused a.on, other things, on the discrepancy between 
the test constructor's .„d outside Judges' perceptions <see Alderson'. 
T.ble 3,. Mhat the paper did not tap .re the perceptions of the 
respondents as to -hat .U„s each it., .as testing. Then of course, 
th. crucial asp.ct is to d.t.r.in. .hat skills .r. jc^u^ 
t.st.d - by ..ans of v.rbal r.port t.chniqu.s. 

Th. .»iscr.pancy b.t...n th. t..t constructor s and outsid. 
iudg..' p.rc.ptions as to th. skills b.ing t.st.d by various it..s 
.i.ply calls att.ntion to th. co.pl.Kity of th. -skill.- ,t 
-ould app.ar that th. t.si constructor and th. t.ach.r J.dg.s had a 
-iH.r.nt s.nse as to -hat it..s t.sting particular r.ading skill. 
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look like. Since part of the problen oay have been in interpreting 
what a given tkill actually entailed, perhaps a useful exerciie Nould 
have been to obtain verbal report data free several of the judges 
regarding the way in which they arrived at their judgeents. It also 
would have been useful to collect the saee kind of data froe the test 
constructor as well. 

The second part of the paper, featuring a case-study nith the two 
respondents P. and J., is excellent — th» kind of work we need euch 
wore of. The fact that little difference was found between introspec- 
tive and retrospective data is not so surprising. The retrospection 
involved "ieeediate retrospection" — within an hour after the test 
taking. P.'s experience of a«aze»ent was in part a reflection of the 
benefits of think-aloud and introspective work — that the inforeants 
get a series of insights about the way that they function in a given 
situation. 

The findings speak loudly about problees of reliability and 
validity in test takingi the difficulty in understanding the test 
itees, the lack of strategies for doing test itees <e.g., P.'s just 
plowing through the text and ^hen reading the questions), the special 
kind of reading evoked in order to find a word appearing as a 
aultiple-choice alternative, the effects of aultiple-choice on finding 
the correct answers, translation problees (e.g., J.'s answer to 19 was 
OK in his native language, Spanish, but probleeatic in English), and a 
aisinterpretation of the instructions (itea 3}. There are also 
instances where the respondents, in fact, displayed th^ expected 
skills (itea S — separating relevant froe irrelevant inforaation, 
itea i — coaparison of two points of view). In the section that the 
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ittOB it«rt taken froe, retpondenti were to reid a tet of quetliont and 
then reread a passage that they had preiueably already read in order 
to answer those questions. Suggested eeans as to hoit to ansiter the 
itees were not provided. The text considered relevant in the ansNer- 
ing of itees nas aleost always specified. 

The conclusions sees quite justified — that the procedures used 
for answering test itees eay not reflect thoie that the test construc- 
tors intended to elicit. It would appear that a test constructor is 
working on a set of hunches as to what will be called for to correctly 
respond to an itee. In reality, there sees to be various weans for 
obtaining an answer — whether correct or incorrect — and with a 
greater or lesser connection to the skill in question. On the basis 
of this study, it would appear thats 

(1) obtaining a correct response way not require the use of the 
designated skill. 

(2) the use of that skill way not necessarily result i^ a correct 
response. 

(3) the answering of a given itew way call for wore than one 
skill. 

(4) an ipforryfX answer is not necessarily an indication that the 
respondent did not possess the skill. The problew could siwply have 
been in interpreting how to do the itew or in an aversion for that 
type of itew (e.g., itew 9, calling for « reading of a graph). 

I would reiterate a point which I raised above, wawwly, that the 
very sawe skill could be realized at different levels of cognitive 
processing. I agree it;th Alderson that wore research is called for. 
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fttvinjti Bodtis qi Rgadinj^ and Rgadino Stratgpigs 

D»vint potitft that thtrt art difftrtnt inttrnaliz«d MdtU of 
rtading which all rtadtri can dtscribt in thtattlvct and which guide 
thM in their rtading. Tht two typts of rtadtrt sht dtalt with 
ttpirically art tht *inttracti vt" rtadtrt and tht "tkillt" rtadtrt. 
Thtftt cattgorizationi art battd both on oral inttrvitui with tht 
rtadtrt in which thty dCBcribt thtir rtading habits and on vtrbal 
rtport protocols obtaintd frot tht rtadtrt at thty rtad a ttltcttd 
ttxt* Htr point it that tht inttractivt rtadtrt art tort tucctttful 
than tht tkillt rtadtrt at cotprthtnding ttxt. 

This ttudy actually builds on two prtvious studits (Otvint 1984, 
Dtvint 19B7)* Nhat D.vint calls an "inttrnal todtl" is actually a 
cotpositt picturt bastd on what rtadtrt say thty do and analysis of 
thtir vtrbal rtport protocols colltcttd while they are in the act of 
reading. Her categorization of readers into "interactive* and 
"skills' readers is derived in part frot work by Hartte k Burke (1977) 
where approaches to reading are referred to as "teaning-centered, * 
*«ord-centered," and "sound-centered." Nith regard to the todel, it 
is worth pointing out that rather than a neat split between intsrac* 
tive and skills readers, there is probably tore of a continuut where 
different readers exhibit both kinds of reading depending on the 
nature of the source text (its length, cotplexity, and coherence), the 
nature of the task (tite alloted, specific instructions given), and 
the background of the reader (schetata, fatiliarity with topic of 
text, expertise in reading strategies). 

For the purposes of this study, an eff was tadt to stltct stu- 
dtntt at tht two tnds of tht sptctrua. Appartrtly tht 14 rtadtrt 
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ititcttd Ntrt drawn fro« an initial pool of 240, fro« which 26 tiere 
•tUctcd and thtn reduced to U bated on stringent criteria. The pit- 
fall of thii kind of saapling it that only extreee catet are keing 
studied, and to the results cannot be said to reflect the patterns of 
the vast eajority of readers. Although the paper does not point this 
out, none of the subjects had received euch specific instruction in 
reading strategies (Devine, Personal Coeeunication) . 

6iver. that reading coeprehension involves an interaction between 
the reader and tae specific text with its features, an analysis of the 
baby talk text for its discourse features would have been helpful. 
Without it, we can only speculate as to what specific features could 
have caused problees. It would be useful to know wore about the 
aeount of the text that the fourteen readers actually coeprehended 
since there can be a discrepancy between learners' iwpres- 
sions/attitudes about how they accoeplish reading tasks and their 
actual perforeance on the tasks. The readers Loepleted a 15-itee eul- 
tiple choice post test and perforeed orai retelling. The author 
reported (Personal CoMunication) that the retelling was not useful 
because of variable handling by the research assistants. The issue of 
what caused difficulties in the text is ieportant since it is possible 
that a skills-based strategy such as focusing on vocabulary could be 
ISre. effective than a top-down/interactive one for dealing with 
certain texts or certain portions of a text. 

It is coeeendable that two reading specialists in Devine's study 
checked all coding decisions. Hention was not eade, however, m to 
Nhcthir thin nai diicrtpancy bitmin thi inviitigator i ratings and 
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thoBt dttsed by tht ipecialists to fat appropriate. It wat aleo not 
clear at to whether the fpecialiftt agreed ifith each other. 

Whereat it is gttting incrtasingly cotton for reading ttrategy 
ttuditt to includt a treattent, in this case the study it non- 
inttrvtntionitt, or ptrhapt, prt-interventioni tt. Tht author con- 
cludtB with tht rtcoattndation that ttachtrt find out tht kinds of 
rtadtrs prtstnt in a givtn class — tost liktly not txtrttts — and 
thtn on that basis dtttrtint tht aid that thty should gtt 
prtsutably aid to skills rtadtrs in hot to bt inttractivt rtadtrt. Of 
courst, it would bt possiblt to argut that inttractivt rtadtrs tay 
alto nttd sott training in hot to itprovt thtir skills in rtading 
t.g., thtir word attack skills. 

In sut, tht author sttts to have found that thost prtdtttrtintd 
stylts Ntrt bornt out by oral rtading ptrfortanct. My lingf^ring quts- 
tiort Nould rtading proficitncy bt consisttnt with tht strattgy 
infortation in this study? Actually, the study is thin on dttail 
about strattgits. Ptrhaps futurt ttudits by tht author could provide 
such dttail to htlp substantiatt tht claits bting #adt. 

Btrnitz and^ Sptaktri Canttxt and Linguistic Structurt 

Barnitz and Sptaktr do a tasttrful job of rtvitning an tnortous 
atount of littraturt rtlating to tht naturt of tht rtading proctss, 
schttata and conttxt, linguistic structurt, and hitrarchical tfftcts 
(11 pagts worth!'/ In thtir rtvitu thty givt a cotpositt dtfinition 
of rtadingt *.«.a tultiltvtl, inttractivt, hypothtsis gtntrating 
proctss in which a rtadtr constructs ttanlng frot a ttxt «ith a cot* 
pltx orchtstration of psycholinguistic proctssts within a particular 
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•ociocultural fituation.* Their rcvieii renindi ub thai ftudiei art 
accumulating in areas which wert prtvioufly laft to fpaculation — 
t.g.9 tht influtnct of cultural background on rtader't ability to 
rocall cultural content in ttxt. They bring us to poetry by indicat- 
ing that tiiis area has been under investigated, thus warranting the 
current study. 

As a aethodological aside, we wight note that the statistical 
procedures (e.g., HANOVA) seews rather sophisticated given the saall 
sawple ~ i.e., 17 seventh graders split into three groups, thus 5-4 
per group. 

Their first findings was that advanced subjects produced sore 
inferential responses than seventh-grade subjects, suggesting to the 
authors that inferential text processing increases with language 
proficiency. Yet the other paper written by the authors on the basi& 
of the saae data (Barnitz ( Speaker 29B7) deaonstrated that 7th 
graders were busy inferencing " just hot inferencing correctly- It 
would be nice to have soae details as to the types of inferences that 
were aade. 

Their third finding was that subjects who produced wore aicro* 
propositions in their recall of text also usid wore linguistic 
aarkers. I wonder whether it would be possible to produce aore aicro- 
propositions and yet fewer linouistic warkeri. The poea is short so 
that the universe of linguistic structures is quite liwited. The lin- 
guistic aarkers are tense, plurals, cohesive ties, and a;reesent. How 
could a aicroproposition be specified without transaitting linguistic 
information of soae kind? Thus, the positive correlation betwten the 
two is not surprising. 

11 
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Ptrhips this if a ainor quibble but the authors refer to *deep 
and surface structures froa the text' as being evident in recall 
protocols. Nhat the authors aean here are 'subordinate and superor- 
dinate propositions,' also referred to as 'aicro and aacro* 
propositions.' This point just reainds us about the probleas of 
terminology. Certainly the teras 'deep and surface structure' call up 
a nuaber of associations totally unrelated to recent Nork in text 
structure. 

The fourth finding was that advanced subjects recalled aore than 
the other two groups. Also, propositions high in the text hierarchy 
were recalled better than those at the aiddle level than those at the 
low level. The authors' surprise at not finding a context effect is 
not all that surprising actually. The context here was not essential 

the topic of the text was interpretable without the need for a 
title. The picture was also not essential. 

The researchers are to be coaaended on their research effort. 
They have identified and incorporated in their study soae of the cru- 
cial variables of concern to researchers of L2 reading. But for 
restarchers interested in cognitive processes, not auch inforaation is 
given regarding application to the ESL reading class. We are reainded 
that it is easier for readers to Aake inferences in a language they 
are eore proficient in. Finally, as the researchers point out in 
their other paper on this saae poea, this is just one poea so its uni- 
verse of eacro- and eicropropositions is liaited. 

RilftiiL, Hiifitfi., Cofdy, ial Hokhtarii Topical Structure Analysis 
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Thii piptr deicribes tm\} ftudiet on recall and hOM to iaprove 
it.* The batic thesis is that 'topical structure analysis* (TSA)' can 
prfivide an important senantic eap for getting through dense acade«ic 
^irofte. 8uc.< analysis is seen to provide the rhetorical structure for 
understanding and reeeMbering text. 

First, there is soee unclarity in the saepling procedures. It 
appears that the control group although eatched on the basis of a 
reading test Nith experieental students, was nonetheless engaged in 
■ore intensive EFL study. Soee of the experimental students were, in 
fact, regular full*ti«e students. Does th*' ^ean they already had had 
the sauie intensive English? Whatever the case, the experimental group 
eas in a writing course and the control group in a course where one 
third of the tiee was devoted to reading. It would be useful to have 
•ore inforeatio.i on the nature of their instruction. Were topic 
sentences and supporting ideas stessed in that course — perhaps 
without being given the TSA nomenclature? 

Furthereore, the experimental group received only four hours of 
instruction on the treatment variable TSA. Also, due to the lack 
of verbal report work, even Just self-report (wherein readers report 
what they do gi»ner ally ) ,^ it is not known whether the experimental 
group actually used their training on the short 12-item multiple* 
choice test that was designed. Also, it is not clear in what ways 
successful performance jn the test was facilitated by TSA. The very 

* I only had access to the write up of one of the studies so my 
remarks are limited to that one. 

> A method of identifying how sentence topics progress from one 
to another, and how the text as a whole relates to the discourse topic 
and subtopicsi Connor 1987. 

^ Other forms of verbal report include self -observation (intro- 
spection and retrospection) and self *reve) ation (think aloud). 
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fact that it Mas in a A^ltiple-choice foriat leant that, as the 
authors point out, it eay not have been 'sensitive enough to reveal 
the hypothesized effect.* 

They suggest that a sueeary nould have revealed comprehension and 
recall eore effectively. Actually, that is not necessarily true. Hy 
ONn recent work with sueearies (Cohen 19B8) would suggest that using 
suaaaries as a testing technique is also probleaatic. A sueaary does 
not necessarily reveal accurately the respondent's level of coaprehen- 
sion. Suaaaries of foreign-language texts aay contain aaterial lifted 
out of the source text with little understanding of its aeaning — 
especially Nhen the suaaaries are uritten in that foreign language. 
In addition, the suaaary aay be of a shotgun nature — an excess of 
detail, Nithin which the correci information is found. On the other 
hand, the suaaary aay be so general and/or vague, that it is not clear 
whether it reflects genuine understanding or not. LikeNise, the 
organization of the text itself (e.g., an obvious set of subheadings) 
aay 'give anay' the topical inforaation. Even if the suaaary does 
reveal nhat the respondent understands about the text, the particular 
rater aay not assess the suaaary accurately, due to a variety of 
intervening factors such as probleas with the respondent's handNrit- 
ing, lack of adherence to the scoring key, and so forth. 

Finally, it is pointed out that recall of seaantic text structure 
was not assessed. Their second study involved the use of recall and 
the results Mere still negative. It is likely that the recall proce- 
dure elicited written texts reseabling in soae nays the saae kinds of 
suaaaries that are critiqued above for their neaknesses. 
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As if evident fro;, the above discussion, the four papers 
addressed here all raise issues of current concern to researchers, 
adainistrators, teachers, and learners. There is a keen desire to 
identify the crucial factors in successful reading coeprehension in 
order to teach reading skills aore effectively. There is a coe- 
■ensurate interest in improving the testing of reading comprehension 
so as to better understand nhat has been attained and what is still 
lacking in the skills of readers. Papers such as these mill continue 
to be valuable as long as the issues reaain unresolved. 
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